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Abstract —Mobile micro-cloud is an emerging technology in 
distributed computing, which is aimed at providing seamless 
computing/data access to the edge of the network when a 
centralized service may suffer from poor connectivity and long 
latency. Different from the traditional cloud, a mobile micro-cloud 
is smaller and deployed closer to users, typically attached to a 
cellular basestation or wireless network access point. Due to the 
relatively small coverage area of each basestation or access point, 
when a user moves across areas covered by different basestations 
or access points which are attached to different micro-clouds, 
issues of service performance and service migration become 
important. In this paper, we consider such migration issues. We 
model the general problem as a Markov decision process (MDP), 
and show that, in the special case where the mobile user follows 
a one-dimensional asymmetric random walk mobility model, the 
optimal policy for service migration is a threshold policy. We 
obtain the analytical solution for the cost resulting from arbitrary 
thresholds, and then propose an algorithm for finding the optimal 
thresholds. The proposed algorithm is more efficient than standard 
mechanisms for solving MDPs. 

Index Terms —Cloud computing, Markov decision process 
(MDP), mobile micro-cloud, mobility, service migration, wireless 
networks 

I. Introduction 

Cloud technologies have been developing successfully in the 
past decade, which enable the centralization of computing and 
data resources so that they can be accessed in an on-demand ba¬ 
sis by different end users. Traditionally, clouds are centralized, 
in the sense that services are provided by large data-centers that 
may be located far away from the user. A user may suffer from 
poor connectivity and long latency when it connects to such a 
centralized service. In recent years, efforts have been made to 
distribute the cloud closer to users, to provide faster access and 
higher reliability to end users in a particular geographical area. 
A notable concept in this regard is the mobile micro-cloud, 
where a small cloud consisting of a small set of servers is 
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Figure 1. Application scenario with mobile micro-cloud. 

attached directly to the wireless communication infrastructure 
(e.g., a cellular basestation or wireless access point) to provide 
service to users within its coverage. Applications of the mobile 
micro-cloud include data and computation offloading for mobile 
devices 0- 0- which is a complement for the relatively low 
computational and data storage capacity of mobile users. It is 
also beneficial for scenarios requiring high robustness or high 
data-processing capability closer to the user, such as in hostile 
environments 0 or for vehicular networks 0J. There are a few 
other concepts which are similar to that of the mobile micro¬ 
cloud, including edge computing |5|. Cloudlet |3j, and Follow 
Me Cloud (§. We use the term mobile micro-cloud throughout 
this paper. 

A significant issue in the mobile micro-cloud is service 
migration caused by the mobility of users. Because different 
micro-clouds are attached to different basestations or access 
points, a decision needs to be made on whether and where 
to migrate the service, when a user moves outside the service 
area of a micro-cloud that is providing its service. Consider 
the scenario as shown in Fig. [T] which resembles the case 
where a micro-cloud is connected to a basestation that covers a 
particular area, and these micro-clouds are also interconnected 
with each other via a backhaul network. When a mobile user 
moves from one area to another area, we can either continue 
to run the service on the micro-cloud for the previous area, 
and transmit data to/from the user via the backhaul network, 
or we can migrate the service to the micro-cloud responsible 
for the new area. In both cases, a cost is incurred; there is a 
data transmission cost for the first case, and a migration cost 
for the second case. 










In the literature, only a few papers have studied the impact 
of mobility and its relationship to service migration for mobile 
micro-clouds. In |7j, analytical results on various performance 
factors of the mobile micro-cloud are studied, by assuming a 
symmetric 2-dimensional (2-D) random walk mobility model. 
A service migration procedure based on Markov decision 
process (MDP) for 1-D random walk is studied in |8j. It mainly 
focuses on formulating the problem with MDP, which can then 
be solved with standard techniques for solving MDPs. 

In this paper, similarly to |8j, we consider an MDP formu¬ 
lation of the migration problem. In contrast to j8), we propose 
an optimal threshold policy to solve for the optimal action 
of the MDP, which is more efficient than standard solution 
techniques. A threshold policy means that we always migrate 
the service for a user from one micro-cloud to another when 
the user is in states bounded by a particular set of thresholds, 
and not migrate otherwise. We first prove the existence of an 
optimal threshold policy and then propose an algorithm with 
polynomial time-complexity for finding the optimal thresholds. 
The analysis in this paper can also help us gain new insights 
into the migration problem, which set the foundation for more 
complicated scenarios for further study in the future. 

The remainder of this paper is organized as follows. In 
Section II, we describe the problem formulation. Section III 
shows that an optimal threshold policy exists and proposes an 
algorithm to obtain the optimal thresholds. Simulation results 
are shown in Section IV. Section V draws conclusions. 

II. Problem Formulation 

We consider a 1-D region partitioned into a discrete set of 
areas, each of which is served by a micro-cloud, as shown 
in Fig. |T| Such a scenario models user mobility on roads, for 
instance. A time-slotted system (Fig. [2]» is considered, which 
can be viewed as a sampled version of a continuous-time 
system, where the sampling can either be equally spaced over 
time or occur right after a handoff instance. 

Mobile users are assumed to follow a 1-D asymmetric 
random walk mobility model. In every new timeslot, a node 
moves with probability p (or q) to the area that is on the right 
(or left) of its previous area, it stays in the same area with 
probability 1 — p — q. If the system is sampled at handoff 
instances, then 1 — p — q = 0, but we consider the general 
case with 0 < 1 ~ p — q < 1. Obviously, this mobility model 
can be described as a Markov chain. We only focus on a single 
mobile user in our analysis; equivalently, we assume that there 
is no correlation in the service or mobility among different 
users. 

The state of the user is defined as the offset between the mo¬ 
bile user location and the location of the micro-cloud running 
the service at the beginning of a slot, before possible service 
migration, i.e., the state in slot t is defined as St = Ut — h±, 
where ut is the location (index of area) of the mobile user, and 
h t the location of the micro-cloud hosting the service. Note 
that St can be zero, positive, or negative. At the beginning of 
each timeslot, the current state is observed, and the decision 
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Figure 2. Timing of the proposed mechanism. 


on whether to migrate the service is made. If migration is 
necessary, it happens right after the state observation, i.e., at 
the beginning of the timeslot. We assume that the time taken 
for migration is negligible compared with the length of a slot. 

We study whether and where to migrate the service when the 
mobile user has moved from one area to another. The cost in a 
single timeslot C a (s) is defined as the cost under state s when 
performing action a, where a represents a migration decision 
for the service such as no migration or migration to a specified 
micro-cloud. The goal is to minimize the discounted sum cost 
over time. Specifically, under the current state so, we would 
like to find a policy n that maps each possible state s to an 
action a = 7 r(s) such that the expected long-term discounted 
sum cost is minimized, i.e.. 


V(s 0 ) = minE 

OO 

so 


.£=0 
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where E [•(•] denotes the conditional expectation, and 0 < 7 < 1 
is a discount factor. 

Because we consider a scenario where all micro-clouds are 
connected via the backhaul (as shown in Fig. [TJ, and the 
backhaul is regarded as a central entity (which resembles 
the case for cellular networks, for example), we consider the 
following one-timeslot cost function for taking action a in state 
s in this paper: 


0, if no migration or data transmission 

/?, if only data transmission 

1, if only migration 

/3 + 1, if both migration and data transmission 

( 2 ) 

Equation Q is explained as follows. If the action a under state 
s causes no migration or data transmission (e.g., if the node and 
the micro-cloud hosting the service are in the same location, 
i.e., s = 0, and we do not migrate the service to another 
location), we do not need to communicate via the backhaul 
network, and the cost is zero. A non-zero cost is incurred when 
the node and the micro-cloud hosting the service are in different 
locations. In this case, if we do not migrate to the current node 
location at the beginning of the timeslot, the data between the 
micro-cloud and mobile user need to be transmitted via the 
backhaul network. This data transmission incurs a cost of /3. 
When we perform migration, we need resources to support 
migration. The migration cost is assumed to be 1, i.e., the 
cost C a (s) is normalized by the migration cost. Finally, if 
both migration and data transmission occur, in which case we 
migrate to a location that is different from the current node 









location, the total cost is j3 + 1. 


be modified to the following: 


Lemma 1. Migrating to locations other than the current 
location of the mobile user is not optimal. 

Proof: We consider an arbitrary trajectory of the mobile 
user. Denote t u as the first timeslot (starting from the current 
timeslot) that the mobile user is in the location indexed u. 
Assume that the user is currently in location u 0 , then the current 
timeslot is t Uo . 

Case 1 - migrating to location u / % at t Uo : This incurs 
a cost of f3 + 1 at timeslot t Uo , because t u > t Uo as a node 
cannot be in two different locations at the same time. Define a 
variable t m £ [ t UQ + 1 , t u ] being the largest timeslot index such 
that we do not perform further migration at timeslots within the 
interval [t Uo +1 ,t m — 1 ], which means that either we perform 
migration at t m or we have t m =t u . Then, we have a cost of 
/? at each of the timeslots t £ [t Uo + 1 , t m — 1 ]. 


[ 0 , if s = 0 

C'a(s) = < /3, if s 7 ^ 0, M < s < N, a = 0 (3) 

[l, if s / 0, M < s < N, a = 1 

With the one-timeslot cost defined as in ([3]), we obtain the 
following Bellman’s equations for the discounted sum cost 
when respectively taking action a = 0 and a = 1 : 


V(s|a = 0) 
V(s|a= 1) 


n'E}=-iPojV(j), 

, 1 + 7Ej=-iPoj^(i), 


if s = 0 

if s 7 ^ 0, M<s<N 

(4) 

if s = 0 

if s 0, M<s<N 

(5) 


where pij denotes the (one-step) transition probability from 
state i to state j, their specific values are related to parameters 
p and q as defined earlier. The optimal cost V ( s ) is 


Case 2 - no migration at t Uo : In this case, the cost at each 
timeslot t £ [t Uo , t m — 1 ] is either fj (if s t = ut ~ ht ^ 0 ) or 
zero (if st = 0). For the timeslot t rn , we construct the following 
policy. If t m < t u , we migrate to the same location as in Case 
1 , which means that the cost at t m cannot be larger than that 
in Case 1. If t m = t u , we migrate to u, which can increase 
the cost at t m by at most one unit compared with the cost in 
Case 1. With the above policy, the costs at timeslots t > t m in 
Cases 1 and 2 are the same. 

The cost at t Uo in Case 1 is one unit larger than that in Case 2, 
and the cost at t rn in Case 1 is at most one unit smaller than that 
in Case 2. Because 0 < 7 < 1, Case 2 brings lower discounted 
sum cost than Case 1. Therefore, there exists a better policy 
than migrating to u / % at t Uo . This holds for any movement 
pattern of the mobile user, and it ensures that the cost in any 
timeslot is either 0 , 1 , or /?. ■ 

From Lemma |T] we only have two candidate actions, which 
are migrating to the current user location or not migrating. 
This simplifies the action space to two actions: a migration 
action, denoted as a = 1 ; and a no-migration action, denoted 
as a = 0. In practice, there is usually a limit on the maximum 
allowable distance between the mobile user and the micro¬ 
cloud hosting its service for the service to remain usable. We 
model this limitation by a maximum negative offset M and a 
maximum positive offset N (where M < 0, N > 0), such 
that the service must be migrated (a = 1 ) when the node 
enters state M or N. This implies that, although the node can 
move in an unbounded space, the state space of our MDP for 
service control is finite. The overall transition diagram of the 
resulting MDP is illustrated in Fig. [3] Note that because each 
state transition is the concatenated effect of (possible) migration 
and node movement, and the states are defined as the offset 
between node and host location, the next state after taking a 
migration action is either — 1 , 0 , or 1 . 

With the above considerations, the cost function in 0 can 




min{F(s|a = 0), V(s|a = 1)}, if M<s<N 
V(s\a = 1), if s = M or s = N 

( 6 ) 


III. Optimal Threshold Policy 


A. Existence of Optimal Threshold Policy 

We first show that there exists a threshold policy which is 
optimal for the MDP in Fig. [3] 

Proposition 1. There exists a threshold policy (ki,k 2 ), where 
M < k\ < 0 and 0 < k 2 < N, such that when k\ < s < k 2 , 
the optimal action for state s is a* ( s) = 0 , and when s < ki 
or s > k 2 , a*(s) = 1 . 


Proof: It is obvious that different actions for state zero 
a( 0 ) = 0 and a( 0 ) = 1 are essentially the same, because the 
mobile user and the micro-cloud hosting its service are in the 
same location under state zero, either action does not incur 
cost and we always have C a (o)(0) = 0. Therefore, we can 
conveniently choose a* ( 0 ) = 0 . 

In the following, we show that, if it is optimal to migrate at 
s = ki — 1 and s = + 1 , then it is optimal to migrate at 

all states s with M < s < k± — 1 or fc 2 + 1 < s < N. We 
relax the restriction that we always migrate at states M and 
N for now, and later discuss that the results also hold for the 
unrelaxed case. We only focus on + 1 < s < N, because 
the case for M < s < ki — 1 is similar. 

If it is optimal to migrate at s = At 2 + 1, we have 


V(k 2 + l|a = 1) < /3y^7* 

t=o 


P 

1-7 


(7) 


where the right hand-side of 0 is the discounted sum cost of a 
never-migrate policy supposing that the user never returns back 
to state zero when starting from state s = k 2 + 1. This cost is 
an upper bound of the cost incurred from any possible state- 
transition path without migration, and migration cannot bring 
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Figure 3. MDP model for service migration. The solid lines denote transition under action a = 0 and the dotted lines denote transition under action a = 1. 
When taking action a = 1 from any state, the next state is s = — 1 with probability q, s = 0 with probability 1 — p — q, or s = 1 with probability p. 


higher cost than this because otherwise it contradicts with the 
presumption that it is optimal to migrate at s = k 2 + 1 . 

Suppose we do not migrate at state s' where k 2 +1 < s' < N, 
then we have a (one-timeslot) cost of 0 in each timeslot until the 
user reaches a migration state (i.e., a state at which we perform 
migration). From 0, we know that V (s\ a = 1) is constant for 
s / 0. Therefore, any state-transition path L starting from state 
s' has a discounted sum cost of 


From 0 and 0 , we know that, for a given policy with 
thresholds (ki,k 2 ), we only need to compute V(s) with 
k\ — 1 < s < k 2 +l, because the values of V(s) with s < k\ — 1 
are identical, and the values of V(s) with s > fc 2 — 1 are 
also identical. Note that we always have k\ — 1 > M and 
k 2 + 1 < N, because k\ > M and k 2 < N as we always 
migrate when at states M and N. 

Define 


t m -1 

V L {s')=0 ]T 7 t + 7 tm V(fc 2 + l|a=l) 

t -o 

where t m > 0 is a parameter representing the first timeslot that 
the user is in a migration state after reaching state s' (assuming 
that we reach state s' at t = 0 ), which is dependent on the state- 
transition path L. We have 

V L (s') - V(k 2 + l\a = 1) 

= F ( fc 2 + 1 |° = 1 ) 

= (1 - 7 * m ) + 1 \ a= ^ 0 

where the last inequality follows from 0 - It follows that, for 
any possible state-transition path L, Vl(s') > V(k 2 + l\a = 1). 
Hence, it is always optimal to migrate at state s', which brings 
cost V{s'\a = 1) = V(k 2 + l|a = 1). 

The result also holds with the restriction that we always 
migrate at states M and N, because no matter what thresholds 
(ki,k 2 ) we have for the relaxed problem, migrating at states 
M and N always yield a threshold policy. ■ 

Proposition [T] shows the existence of an optimal threshold 
policy. The optimal threshold policy exists for arbitrary values 
of M, N, p, and q. 


B. Simplifying the Cost Calculation 

The existence of the optimal threshold policy allows us 
simplify the cost calculation, which helps us develop an al¬ 
gorithm that has lower complexity than standard MDP solution 
algorithms. When the thresholds are given as (fci, k 2 ), the value 
updating function ([ 6 ]» is changed to the following: 

V( S ) = i V ^ a = 0 )’ iffc l< S < fc 2 g 

| V(s|a = 1), otherwise 


P', 


(ki,k 2 ) 


[ki - 1) P(fci) • • 

•no)- 

• V{k 2 ) V(k 2 

+ 1)] T 
(9) 

[1 0 ■■■ 0 

0 

•• 0 1 ] T 


-- v- - S 



(10) 

— k\ elements 

k 2 elements 

P0,fci-1 

Poo 

P0,fc 2 + 1 


Pki ,ki~l 

Pk u 0 

Pk\,k 2 +1 


P 0 M -1 

Poo 

P0,fc 2 + 1 

(11) 

Pk2,ki — 1 

Pk 2 ,0 

Pk 2 ,k 2 +1 


_ P0,fci-1 

Poo 

P0,fc 2 + 1 . 



where superscript T denotes the transpose of the matrix. 

Then, |4]) and |5]i can be rewritten as 

v (fci,fc2) = c (fci,fc 2 ) + 7 p (fe 1 ,fc 2 ) v (fei,fc 2 ) (12) 


The value vector V( fcl k2 ) can be obtained by 

v (fei,fc 2 ) ~ (l — 7P(fci,fe 2 )) c (fci,fc 2 ) (12) 

The matrix ^1 — 7P(fc 1 is invertible for 0 < 7 < 1, 
because in this case there exists a unique solution for V( fel fe2 ) 
from ( fl2l >. Equation ( p~3| ) can be computed using Gaussian elim¬ 
ination that has a complexity of O ^(|M| +N) 3 ^j. However, 
noticing that P' fc| k ^ is a sparse matrix (because pij = 0 for 
\j—i\ > 1 ), there can exist more efficient algorithms to compute 

m 

C. Algorithm for Finding the Optimal Thresholds 

To find the optimal thresholds, we can perform a search 
on values of (ki,k 2 ). Further, because an increase/decrease in 
V(s) for some s increases/decreases each element in the cost 
vector v due to cost propagation following balance equations 












Algorithm 1 Modified policy iteration algorithm for finding 
the optimal thresholds 

1 

Initialize 4— 0, k 2 0 


2 

repeat 


3 

k'f 4 — k\, k 2 4 — k 2 //record previous thresholds 

4 

Construct c/.» ,,\ and P', , according to 

__ 2 ) ^k 1 ,k 2 ) 

CD 

| 10 jl and 

5 

6 

Evaluate v/ fe » k from (JT3|) 

Extend v^ k , fc „^ to obtainV(s) for all M < s 

< TV 

7 

for i = 1,2 do 


8 

if i = 1 then 


9 

if 1 + 1 52^-1 P 0 jV(j) <V(k$) then 


10 

dir 4 - 1, loopVec [k\ + 1, fc* + 2,... 

,0] 

11 

k^ ^k{ + 1 


12 

else 


13 

dir 4 — 0, loopVec [k* — 1, fc* — 2,... 

,M + 1] 

14 

end if 


15 

else if i = 2 then 


16 

if 1 + 'yE]=-iPo j V(j) < V(k* 2 ) then 


17 

dir t— 1, loopVec 4 — [k 2 — 1, k 2 — 2,... 

,0] 

18 

k 2 4r- kl - 1 


19 

else 


20 

dir t— 0, loopVec [k 2 + 1, k 2 + 2,... 

, TV — 1] 

21 

end if 


22 

end if 


23 

for ki = each value in loopVec do 


24 

if dir = 0 then 


25 

if ^ + 'rT l j=k i -iPk i ,jV(j) < V(ki) then 

26 

k* i — ki 


27 

else if /3 + 7 YlfJkl-iPkijVU) > V (k t 

) then 

28 

exit for 


29 

end if 


30 

else if dir = 1 then 


31 

if 1 + 7 J2j=-iPojV(j) < V(ki) then 


32 

k* 4 h - sign (ki) 


33 

else if 1 + 7 PojV(j) > V(ki) 

then 

34 

exit for 


35 

end if 


36 

end if 


37 

end for 


38 

end for 


39 

until kl = k'f and k 2 = k 2 


40 

return k\, k 2 



0 and (|5jl, we only need to minimize V(s) for a specific state 
s. We propose an algorithm for finding the optimal thresholds, 
as shown in Algorithm [T] which is a modified version of the 
standard policy iteration mechanism [9j Ch. 3]. 

Algorithm [T| is explained as follows. We keep iterating until 
the thresholds no longer change, which implies that the optimal 
thresholds have been found. The thresholds (k\, k 2 ) are those 
obtained from each iteration. 

Lines |4j|6] compute V(s) for all s under the given thresholds 
(k*, k 2 ). Then, Lines [ 8 ]- 22 determine the search direction for 
fci and £ 2 - Because V(s) in each iteration is computed using the 


current thresholds (k\,k 2 ), we have actions a(fe^) = aik^) = 
0 , and 0 is automatically satisfied when replacing its left 
hand-side with V(k*) or V(k 2 ). Lines |9]and 16 check whether 
iterating according to (5) can yield lower cost. If it does, it 
means that migrating is a better action at state (or k 2 ), 
which also implies that we should migrate at states s with 
M < s < fc* (or k 2 < s < TV) according to Proposition [T] 
In this case, k\ (or k 2 ) should be set closer to zero, and we 
search through those thresholds that are closer to zero than 
/,:* (or k 2 ). If Line [9] (or Line 16 1 is not satisfied, according 
to Proposition [T] it is good not to migrate at states s with 
fc* < s < 0 (or 0 < s < k 2 ), so we search ki (or / 02 ) to the 
direction approaching M (or TV), to see whether it is good not 
to migrate under those states. 

Lines |23](37] adjust the thresholds. If we are searching toward 
state M or TV and Line [25] is satisfied, it means that it is better 
not to migrate under this state (ki), and we update the threshold 
to ki . When Line [27] is satisfied, it means that it is better to 
migrate at state ki. According to Proposition [I] we should also 
migrate at any state closer to M or TV than state ki, thus we 
exit the loop. If we are searching toward state zero and Line|3T| 
is satisfied, it is good to migrate under this state (ki), therefore 
the threshold is set to one state closer to zero (ki — sign(fci)). 
When Line [33] is satisfied, we should not migrate at state ki. 
According to Proposition [T] we should also not migrate at any 
state closer to zero than state ki, and we exit the loop. 


Proposition 2. The threshold-pair (fc*, k 2 ) is different in every 
iteration of the loop starting at Line [2] otherwise the loop 
terminates. 


Proof: The loop starting at Line [2] changes k\ and k 2 
in every iteration so that V"(s) for all s become smaller. It is 
therefore impossible that k\ and k 2 are the same as in one of 
the previous iterations and at the same time reduce the value of 
V(s), because V(s) computed from (13 1 is the stationary cost 
value for thresholds (k^,k 2 ). The only case when (k\,k 2 ) are 
the same as in the previous iteration (which does not change 
V{s)) terminates the loop. ■ 


Corollary 1. The number of iterations in Algorithm [7] is 
0(\M\N). 


Proof: According to Proposition [2] there can be at most 
\AI\N + 1 iterations in the loop starting at Line [ 2 ] ■ 

If we use Gaussian elimination to compute (13) , the time- 
complexity of Algorithm [I] is O (|M|TV(|M| +TV) 3 ). 

IV. Simulation Results 


We compare the proposed threshold method with the standard 
value iteration and policy iteration methods 0 Ch. 3], Simula¬ 
tions are run on MATLAB, on a computer with 64-bit Windows 
7, Intel Core i7-2600 CPU, and 8 GB memory. The value 
iteration terminates according to an error bound of e = 0.1 
in the discounted sum cost. Note that the proposed method and 
the standard policy iteration method always produce the optimal 
cost. The number of states \M\ = TV = 10. The transition 
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Figure 4. Performance under different 0 

probabilities p and q are randomly generated. Simulations are 
run with 1000 different random seeds in each setting to obtain 
the average performance. The running time and the discounted 
sum costs under different values of /? are shown in Fig. [4] 

The results show that the proposed method always has lowest 
running time, and the running time of the standard policy 
iteration method is 2 to 5 times larger than that of the proposed 
algorithm, while the value iteration approach consumes longer 
time. This is because the proposed algorithm simplifies the 
solution search procedure compared with standard mechanisms. 
The results also show that the proposed method can provide the 
optimal cost compared with a never-migrate (except for states 
M and N) or always-migrate policy. It is also interesting to 
observe that the optimal cost approaches the cost of a never- 
migrate policy when /3 is small, and it approaches the cost of an 
always-migrate policy when /? is large. Such a result is intuitive, 
because a small (3 implies a small data transmission cost, and 
when /3 is small enough, then it is not really necessary to 
migrate; when j3 is large, the data transmission cost is so large 
so that it is always good to migrate to avoid data communication 
via the backhaul network. 

V. Conclusion and Future Work 

In this paper, we have proposed a threshold policy-based 
mechanism for service migration in mobile micro-clouds. We 
have shown the existence of optimal threshold policy and 
proposed an algorithm for finding the optimal thresholds. The 
proposed algorithm has polynomial time-complexity which 
is independent of the discount factor 7. This is promising 


: (a) 7 = 0.5, (b) 7 = 0.9, (c) 7 = 0.99. 

because the time-complexity of standard algorithms for solving 
MDPs, such as value iteration or policy iteration, are generally 
dependent on the discount factor, and they can only be shown 
to have polynomial time-complexity when the discount factor 
is regarded as a constanfj] (10|. Although the analysis in this 
paper is based on 1-D random walk of mobile users, it can serve 
as a theoretical basis for more complicated scenarios, such as 
2-D user-mobility, in the future. 
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